SYSTEM, METHOD AND SOFTWARE FOR INDIVIDUALS TO EXPERIENCE AN 
INTERVIEW SIMULATION AND TO DEVELOP CAREER AND INTERVIEW SKILLS. 

RELATED APPLICATIONS 

10001] This application claims priority to U.S. Provisional Patent Application Serial 

Number 60/442,669, filed on January 27, 2003, the disclosure of which is herein incorporated by 
5 reference. 

FIELD OF THE INVENTION 

[0002] The present invention relates in general to the field of interactive software and 

more particularly to a system, method and software for providing interactive employment 
10 interviews, automated employment screening, employment interview training, speech training, 
career training, and employment interview preparation. 

BACKGROUND OF THE INVENTION 

[0003] Organizations spend excessive amounts of time and money interviewing 

15 candidates for employment. Although they use a variety of timesaving techniques such as phone 
interviews and paper exams, these techniques do little to curb the high cost of interviewing 
candidates. Moreover, staffing agencies boast of providing value-added services, but in the end 
only provide resumes with practically no useful verification. Candidates also spend a 
tremendous amount of time searching and interviewing for jobs, yet often find that they are 
20 either unlikely to secure the position because they are unqualified for the position or they do not 
wish to pursue it. 

[0004] There is, therefore, a need for a system, method and software for organizations to 

automate the process of interviewing and screening candidates. The present invention allows 

25 organizations to process candidates through an automated interviewing tool that can determine 
which are the best candidates to bring in for live interviews. There is also a need for a system, 
method and software for candidates to pre-screen themselves to determine which jobs to apply 
for and to create additional resources for candidates to market themselves to employers. The 
present invention allows individuals to perform virtual interviews that can be analyzed for 

30 qualifications and submitted to employers for screening purposes. 
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[0005] Furthermore, in an increasingly competitive job market where candidates share 

similar skill sets and experience, the interview becomes the deciding factor in the hiring process. 
In the current environment, individuals do not have the means to sufficiently practice job 
interviews. At best, individuals can practice interviews with a live person. However, most 
individuals have very limited access to such a person due to cost, time and availability 
constraints. Inferior substitutes include interview question books, online sites with generic 
questions, interview tactics workshops, interview videos, and computer based training for a 
particular skill set. 

[0006] There is, therefore, a need for a system, method and software for individuals to 

rehearse their interviewing skills. The present invention allows individuals to practice, develop, 
and refine their interviewing skills. Individuals can practice an interview as many times as they 
wish from any location with access to a computer. 



[0007] Previous patents have focused on the ability to communicate in text and in speech 

with a computer, interactive learning, virtual characters, synthesized speech and expert systems, 
but no patent combines these concepts and/or new concepts into a system, method and software 
for interactive employment interviews used for screening and training. The present invention 
20 solves the need for this technology. 

SUMMARY OF THE INVENTION 

[0008] The present invention relates to interactive software and provides a system, 

method and software for individuals to experience an interview simulation. It allows 
25 organizations to create generic and job specific interviews that can be administered in an 
automated manner to job applicants for screening purposes. The present invention also allows 
job seekers to screen themselves and provide pre-screened interview data to employers. Finally, 
the present invention provides a means for individuals to develop career and interview skills by 
learning about and practicing for generic and job-specific interviews. 

30 
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10009] Interviews can be conducted locally, they can be conducted remotely by utilizing 

a remote server computer. Interviews can be conducted on a computer or any other device that 
can process the software. Such devices may include one or more of the following input/output 
devices, keyboard, microphone, video camera, web camera, sound card, video card, modem 
connection, network connection, local area network connection, metropolitan area network 
connection, wide area network connection, intranet connection, and wireless network connection. 

[0010] The system, method and software utilize pre-interview and post-interview data 

that is incorporated into the interview simulation and analysis. Examples include but are not 
limited to the resume, employment application, choice of character, clothing, job research, 
traveling, interpersonal interactions within a company, salary negotiations, and post interview 
correspondence. 

[0011] The system, method and software allow individuals to communicate with one or 

more software-generated animated interviewers. Communication is bi-directional. The software 
can speak to the individual by displaying statements on the computer terminal and/or digitizing 
output into sound. The individual responds to the interviewers by typing and/or speaking 
statements into a device such as a microphone or video camera that records and translates the 
responses into the system. 

[0012] The system, method and software are able to simulate an interview conversation 

based on a dynamic interview plan and internal expert system. This allows a user to experience a 
series of interconnected discussions that create an interview discussion as a whole. 

[0013] The system, method and software are capable of producing a large number of 

generic and job-specific questions related to the type of interview in which the user has chosen. 
These questions can also be proposed in response to previous interview questions and responses. 

[0014] The system, method and software provides detailed screening, review, analysis, 

and feedback for all stages of the interview simulation and displays results using a customized 
computer interface on the computer terminal. The screening and analysis evaluates all input 
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including pre-interview, interview, post-interview, explicit and implicit data. Screening and 
analysis also produces a series of recommendations based on the interview interaction. The 
recommendations can be provided to hiring managers for screening purposes, or directly to the 
user if used for training puiposes. The format of the recommendations can change based on the 
5 needs of the organization and user. The system also suggests additional external help resources 
based on the needs of the user and uses an algorithm to match the needs of the user with a 
database. 

[0015] The system allows for full customization of the interview simulation, either for 

10 screening or for training. This includes but is not limited to the editing and configuration of 
company information, interview rooms, interviewer profiles, job information and requirements, 
classified ads, interview agendas, testing data, and industry knowledge. 

BRIEF DESCRIPTION OF THE DRAWINGS 

1 5 [0016] These and other features, aspects, and advantages of the present invention may be 

better understood by referring to the following description in conjunction with the accompanying 
drawings in which: 

[0017] FIG 1. is a block schematic diagram which outlines how the employment 

20 interview system is composed of sub systems and databases. 

[0018] FIG 2. is a block schematic diagram which gives insight into how text, speech, 

graphics, and environment events interact. 

25 [0019] FIG 3. is a block schematic diagram which explains how expert systems can 

cooperate to implement a job interview discussion simulation. 

[0020] FIG 4. is a block schematic, diagram that outlines how a user chooses a job and 

how the job data is used to drive the interview simulation. 

30 
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(0021] FIG 5. is a block schematic diagram that outlines how job-seekers and employers 

utilize the system to find each other. 

[0022] FIG 6. is a block schematic diagram that displays how different types of clients, 

5 including different communication protocols and platforms, are supported by the system. 

[0023] FIG 7. is a block schematic diagram that explains how the employment interview 

system can be extended beyond job interviews with other types of knowledge and information. 

10 [0024] FIG 8. is a block schematic diagram that displays how the employment interview 

system transmits interview data to the system and employers. 

[0025] FIG 9. is a block schematic diagram which explains how the interview system 

supports telephone, Voice over IP (VoIP) and video phone clients. 

15 

[0026] FIG 10. is a block schematic diagram that displays how the interview system can 

be administrated remotely and how interviews can be coordinated by live interviewers. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
20 [0027] Introduction to the Employment Interview System: The high-level employment 

interview system is seen in FIG 1. This system has an input system (101) that is responsible for 
receiving and managing input from the user. This input can be in the form of text, speech, video 
data, or hardware events such as mouse or keyboard actions. Not all input is in the form of 
communications. Some input can be in the form of a control event, such as asking the 
25 interviewer to proceed to the next question, or having a virtual character express sadness during a 
salary negotiation. The input data consisting of video data can be a live video feed of the user 
speaking and reacting to the interviewer, exactly as in what would be expected of in a real job 
interview. The user may be able to speak into the system using a microphone. The speech data 
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can be processed in a variety of ways. First, the speech data may be used in its original form to 
be stored and reviewed later by the user or other interested parties. The speech data may also be 
streamed into a speech recognition system, followed by syntax, and application domain 
tweaking, and then fed into a natural language parser to extract desired input phrases. The output 
5 system (102) includes the visual aspects as well as the audio aspects of the interview. The visual 
aspect may include a direct video feed from a remote interviewer, or a computer generated 
representation of one or more interviewer characters. When using a computer generated scene it 
is likely that the user will also be able to see environments such as an interview room and desk. 
The audio aspects include voices of the interviewers as well as closed captioning text if desired. 

10 The system logic (103) utilizes a set of logic routines to manage the interview discussion. These 
discussion management routines utilize a set of specialized state machines and expert systems for 
various aspects of the interview. Though they cannot handle all conversations perfectly, they do 
have enough logic to handle a wide range of interview discussion topics when supported by 
appropriate databases. The two key databases are the job knowledge database (104) and the 

15 language database (105). The job knowledge database contains information about job 
descriptions, human resources, and job specific information such as skill files, which contain 
questions, answers, analysis, and scores. The language database contains language specific 
information such as dictionaries, synonyms, pronunciation rules, and other information related to 
natural language processing. Finally, depending on the exact use of the system it is possible to 

20 have a communications subsystem (106), which would allow an interviewer to be detached from 
the interview system This configuration may be useful when the user of the interview system is 
on a telephone, videophone, or a remote computer on a network. 
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[0028] Interview System for Employers: Employers may want to directly incorporate the 

interview system to help interview corporate applicants. The employer may be a direct employer 
or an intermediary employment agency that is seeking to identify qualifying candidates. In 
either case the employer may use the system to interview candidates. The system can be 
5 configured in such a way that the employer provides the job knowledge including an interview 
agenda plan and specific questions and skills to discuss. The applicant can use the system over 
the phone or through a computing device. The applicant may be local or at a remote site. The 
interview system will also allow an employer to directly control the interview with an 
administration tool (1003), which will allow a person at the employer to have full control of the 

10 interview discussion, and if necessary switch between an automatic interview using the expert 
systems and a manual interview with the employee speaking or typing into the administration 
tool. If the system is used to interview the candidate, the employer will receive an analysis of the 
applicant's performance based on information found in the job knowledge database as well as 
other non-qualitative information such as ability to answer quickly, ability to communicate 

15 effectively, and interpersonal skills. The system analysis can be viewed immediately by an 
administrator or viewed sometime in the future in the form of a report or email. 



[0029] Interview Matching System: The system can be configured in a manner in which 

the system could have the ability to match job seekers with job opportunities. FIG 5, depicts a 

20 matchmaking system based on the interview system presented herein. At the core of the match 
making system is the interview system (502), which may take form as an interview system 
server. The job candidate (501) will choose and go on a job interview for a well known job type 
or a specific open job position. Employers (503) may post job openings or may just scan the 
results (504, 505) of specific job seekers. Job seekers who submit their interview information 

25 when applying for a job will provide the interview system with general user data such as resume 
and background information (504). In addition, job seekers will provide interview results after 
each interview such as transcript, audio, video, and analysis. The interview match system also 
has a database with employer job descriptions (506). The employer job description database 
contains job ads and job descriptions with triggers to contact the employer if a candidate has 

30 qualified. For example, if an employer creates a job, the employer may want to be notified by 
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email if an accountant has interviewed and has passed the minimum score for two of the five key 
skills in the specified job description. 

[0030] Interview Training System: The interview system described herein lends itself to 

5 career development applications, in particular job interview training. The system can be used to 
provide practice job interviews. Several different types of interview training sessions can be 
made from the base interview system. First, a user can choose from the available jobs and go on 
a job interview. Second, the user can build a job interview based on a set of job criteria that the 
user selects. Third, the user may desire training in one aspect of a job interview, and the system 

10 can provide specific training in only that job area. Finally, the interactive training program will 
have access to all of the input and output systems of an interview training application, allowing a 
user to record mock interviews with another live interviewer. Since the training system has 
access to the job knowledge base, the system could allow the user to prepare for job interviews 
with information about common questions based on the job desired along with the user's 

15 experience, education, skills, and goals. The system could also show the user recommended 
answers when the user reviews an audio or video recording of practice interviews. As a training 
platform, users could also become familiarized with the stages of professional interviews, such 
as choosing travel options, traveling to the location, entering the corporate site, reception area or 
lobby, filling out an application, meeting with the human resources department, walking to the 

20 interview room, interviewing, sending post interview thank you notes, handling second 
interviews, and handling salary discussions. The training system can provide the user with 
information after textually analyzing a job application, cover letter, or resume. Since the system 
has a job description in the job knowledge database, the job description can highlight the skills 
and traits required, minimum level of education, and minimum level of experience. The training 

25 system could also not only provide a localized language user interface and help system, but it 
could also provide multilingual interviews based on the language database that the interview 
system utilizes. It is important to note that the interview training application can work on a 
standalone machine as well as in a network or Internet environment. The application may also 
be built on a wide range of languages such as C, C++, Java, Shockwave Lingo, C#, Perl, Visual 

30 Basic, and others with similar or additional capabilities. The operating systems could also be 
vast such as personal computer operating systems and embedded operating systems, as long as a 
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suitable input/output system and associated interview system code can exist, or can be reached 
through a communications medium such as TCP/IP. 

[0031] Rendering Interview Representation: Although the interview system is fully 

5 functional without a sophisticated graphics system (i.e., text based), a sophisticated graphics 
system could be used in conjunction with the interview system. Interviewer characters can be 
rendered in 2D (composite images), or 3D environment (3D objects in a space with configurable 
points of view). Certain applications may choose to render the interview characters with 
photorealistic imagery and others with less realistic animated cartoons. In either case, the 

10 invention will support a range of artistic mediums. In order to achieve animation the interview 
system will trigger a set of events to notify the animation system of character and sub-character 
states. The character states can be used to choose the appropriate graphics image or rendering. 
Sub-character states allow characters to move different body parts at the same time; for example 
the lips can be set to one state, while the body is set to another state. All character animation 

15 states are represented with a list of numbers or distinct labeled strings. The interview system 
determines what interviewers will say, how characters will say certain things, how characters 
will interpret and react to user input, how characters feel, and what high level actions characters 
should be performing. A sophisticated graphics system can take information from the server and 
render it (FIG 2, 201) for the particular interviewer. The system may also control and render 

20 body actions; for example, looking around the room, and nodding to input when a user is talking 
or typing. The system has information about a virtual interviewer such as happiness and interest 
level, so that when the application is in an idle state, it may render an appropriate manager 
emotional state. The user may be rendered in 2D (composite images), or 3D environment (3D 
objects in a space with configurable points of view). The user view may not be in the view in the 

25 case when the user is taking a first person view. The user may be partially viewed such as in the 
case when the camera is over the user's shoulder, in which case the display will show the back of 
the head, body, and possibly hands of the user. In a 3D environment the user may or may not be 
fully viewed depending on the camera angle within the room. The view (i.e., camera angle and 
location) can be chosen automatically by a smart software camera manager based on the location 

30 of key characters, along with a collection of preferred camera positions. The view may also be 
selected manually by the user. Common views include first person, side view, and top view. 
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The best view may also depend on the number of characters in the interview scene, when for 
example there is one interviewee and three interviewers in a corporate conference room. The 
user may have the choice to select and build a character to use for the interview. This may 
include visual and non-visual attributes. Visual attributes include gender, body type, skin color, 
5 hah- color, type and amount of jewelry, clothing style, clothing colors and patterns, and others. 
Non-visual attributes may include cologne and perfume, and others. In certain application 
modes, the user will have the ability to control the character including body position, head and 
body gestures, and facial expressions. Facial expressions will help provide an additional level of 
control by allowing a user to show happiness, enthusiasm, disappointment, and other emotions 

10 that may be required during an interview. The user will have some control of explicit actions, 
but may have implicit control over others, such as when a user is talking into a microphone and 
has configured his or her character to use hand gestures, in which case the client system will 
automatically move hands in an appropriate manner while the user speaks. The interview 
rendering may utilize a simple background image, animated video background, or 3D model 

1 5 rendering, or a more advanced 3D rendering with animated textures. The job knowledge sent to 
the interview system could be used to deteimine the appropriate interview room environment, 
since information about the industry and company are available. Some examples of interview 
environments are a small office, conference room, and interview room in a human resources 
department. Environments can be used to provide a richer visual interview experience, such as 

20 when the user is able to see scenes before the interview such as the waiting room, or after the 
interview such as a company tour. 



[0032] Management of Interview Data: When experiencing the most realistic form of 

interview, the user may choose to provide the system with detailed background information such 

25 as what is typically found in a job application or resume. In addition, when additional hardware 
is available, the interview system will have the capability to record audio through a microphone, 
and record video through a web camera or standard video camera. Depending on the use of the 
interview system, an interview analysis may also be available. In aggregate, the specific 
interview information will consist of background information, audio data, video data, and 

30 analysis. The specific interview information can be recorded and saved locally or remotely 
depending on the need. Saving the data remotely can be done in a file system or by using a 
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network medium. The information may also be digitized, especially when recording multimedia 
signals. It can also be compressed using a proprietary or standard compressor for the multimedia 
data. In addition, the multimedia data may be combined into one digital data stream, instead of 
an audio and video stream. Although combined, the data stream can use two distinct 
5 compression algorithms or one algorithm. The system does not require any particular file format 
or compression standard, and thus is flexible in that respect. The specific interview information 
can also be encrypted with a user or system provided key and algorithm. The specific 
information may be saved and indexed to be reviewed or compared later. It is also possible for 
the specific interview information to be reviewed with by others in real-time or at a later time. 
10 Other interested parties may include advisors and employment agencies, and of course should be 
done in a way consistent with the rights of the user. 

[0033] Transmitting Interview Data: As alluded to earlier, the interview system can be 

used to transmit the content and results of the interview to a remote location. The content could 

15 be a real-time audio or video stream to an interested party, such as an employer with an open 
position. FIG 8, demonstrates how a real-time interview client (801) is sending interview data to 
an interview server (807). The employer (806) or other party's system can then access the 
interview data (805) through the interview server. The client may send real-time data because it 
is the desired mode of operation, or because it is incapable of storing local data. Other clients 

20 (802, 803) may have various amounts of local storage and may choose to temporarily or 
permanently store interview data locally. An enhanced system could utilize a wide range of 
networking protocols to move data from the user application. In certain configurations such as 
FIG 9, it is possible to have telephone based interviews stored on the server, in which case this 
data could be retransmitted or converted to another format such as text transcript, and then 

25 retransmitted to an interested party. Retrieval of interview data is not only possible by third 
parties such as the employment agency, it is also possible by the interview clients (801, 802, 803, 
901, 902, 903) when necessary. 

[0034] Communications and Control: When the system logic is directly connected to the 

30 user interface, the communications layer acts as a pass through mechanism. However, when the 
system logic is remotely connected to the user interface, the two components incorporate a 
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communications layer FIG 6, (607). The client and server communicate using messages. 
Messages are a platform independent payload that can contain a wide range of data such as 
strings, text, and binary. The messages can be transmitted over a wide range of communication 
mediums and protocols. They can be used on connection oriented systems such as TCP/IP and 
5 non-connection oriented systems such as an IPX network. Similarly, the system can be used 
over wired or wireless systems. The messages contain general information such as type and 
version information as well as a collection of message data. The most common messages 
contain control codes or data. Some control messages manage the communications session, such 
as logon to server, and disconnect from server. Some control messages handle pre-interview 

10 data such as send user information and request job information. Some control messages handle 
interview specific messages such as start interview, end interview, send action, and send data. 
Some control messages are for post interview events such as submit post-interview data and get 
interview results. Messages may be passed in a plaintext, encrypted, compressed, encrypted and 
compressed, other binary or text formats depending on the configuration. FIG 2, shows how the 

15 server (206, 207) is able to send and receive a wide variety of speech and action events. The 
system utilizes text messages that contain control codes and data. Some of the messages contain 
speech messages represented with text characters. The client application (204) may type some 
text (202) that will be sent to the server as user input. The client application may also use a 
speech recognition component (203) that will convert speech to text, do some additional 

20 language processing, and then send the text to the server. The client application may also send 
pure speech to the server, and let the server handle the speech recognition process. The best 
formula depends on the capabilities and needs of the client and server. The server is able to 
generate speech messages from the hiring managers and send them to the client as audio speech 
messages or text messages. The client will then either show the text as closed-captioned text 

25 (202), or render the text via a text to speech component (203). Speech messages may also 
contain clues that may alter the modulation of speech or trigger facial or body emotions or 
gestures. For example text can contain an exclamation point to signal excitement. In addition, a 
text message could contain a code such as <disappointment> within a text string such as "I'm 
sorry that is wrong." resulting in a manager character speaking and showing disappointment at 

30 the same time. 
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[00351 An important aspect of the messaging system is that it allows a local client or 

remote client with a system server to use a set of inter-connected message pipeline components 
for input and output. The pipeline infrastructure and components support transformation and 
multiple forms of data to communicate. For example the user input can communicate with the 
5 interview system in a variety of ways such as speaking with text input (202) or speaking with 
voice (203). The text can be packaged into one or more messages and then transferred to the 
system. The voice data can be packaged into one or more messages and then transferred to the 
system, unpacked and then processed through a variety of additional information transformation 
engines such as a speech recognition system to convert audio to text that can be parsed by an 

10 interview discussion engine. There may also be the configuration in which the client application 
may want to convert the speech to text on the local side, then use text for discussion messages 
which are sent to the system server for further processing as user input. The user output system 
also is controlled by message oriented control and data. For example, the system may send the 
client a phrase that an interviewer wishes to ask. In the case where there are multiple 

1 5 interviewers in an environment, the phrase will also be accompanied by a unique interviewer ID. 
The user output system may receive the phrase in the form of a text phrase embedded within a 
message. The client system (204) may decide to additionally render the text through a text to 
speech engine to supplement displayed text or replace the phrase spoken by the interviewer. The 
client platform may not be capable of rendering the text to speech message, in which case the 

20 client may ask the server to render the speech for it, and send it the audio stream of the 
interviewer phrase in addition to other text information such as lip-syncing information, phrase 
text, and interviewer ID. FIG 3, shows how the discussion system has access to the input and 
output queues, and has a wide variety of helpers to work with the queues. For example the 
expert system may want to know how long has passed since the interviewee spoke last, and may 

25 refer to (304, 305). The input/output queuing mechanism can support multiple client sources and 
targets. The expert systems can retrieve the spoken words, whether the words were sent as text 
or speech. The system logic in (612) will have the ability to pre-process messages upon 
receiving them prior to placing them in the system input/output queues for retrieval from the 
expert systems. The discussion system can also use and transform a set of output data messages 

30 and control messages. This may be based on the client's preferences or limitations. One 
particular case is when the system server sends the client text, text and audio speech data, or 
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speech audio data. This capability-limitations-preference model can also be applied to video, 
where a system server may send the client a graphics or video stream containing a configurable 
stream of renderings during an interview experience. This situation would require no art or 
sophisticated client-side graphics sub system. Alternatively clients may decide not to render 
graphics at all, or request that the server system send the client control messages so that a client 
may render an interview locally in text, 2D, or 3D. The control messages could contain specific 
environment events or transitional updates such as interviewer character #1 is nodding her head 
up and down. 

[0036] Execution on Standalone or Network Device: The interview system presented can 

be implemented entirely on one machine, or can be partially implemented as an interview client 
with reliance on an interview server, which will handle the remaining system logic. FIG 8, 
demonstrates how clients with different memory capabilities can access the interview server. 
The same principle can also be used for different client systems with little to advanced input 
systems. In the most simple input system, an interview training session can skip actually 
answering questions, and simply trigger an input event to proceed. Systems that have a little 
more capability such as having a few buttons or a small range of inputs, can use those inputs to 
answer multiple choice questions. More advanced systems will have keyboards or simulated 
keyboards, in addition to audio input and speech recognition capabilities. In many cases, the 
interview server can supplement a lightweight client by either doing work for the client or 
providing the client with appropriate data for that platform. The graphics interface of a network 
client (605) may also have a range of capabilities that can be supported by an interview server. 
The design of the system lends itself to be used by a wide range of computing platforms, such as 
standard PCs, laptop PCs, dummy terminals, kiosks, Personal Digital Assistants, and mobile 
phones with application support. 

[0037] Interview client applications can be programmed on a variety of programming 

languages, and can function on a variety of operating systems. Network clients can use a variety 
of communications mediums (607) such as wireless and wired networks. Though some networks 
will have higher capabilities, for example current limitations do not effectively support video 
streaming over a wireless network, though it is currently possible by the client and server, as can 
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be achieved over a LAN or common home Internet broadband connection. Interview clients and 
servers can use a variety of communications protocols to communicate. For example, the clients 
can use IP and IPX. Some protocols, such as IPX and UDP, may require additional protocol 
layers to guarantee data, order, and manage sessions. The clients and servers can also support 
5 higher level protocols such as TCP/IP and HTTP over TCP/IP. As long as the client and server 
support the same protocol, different types of network clients can use the interview server system 
services. A wide range of communications mediums (607) or networks can be utilized to 
provide a computer-based interview. Some of the many possible client/server configurations 
include modem to modem, modem to intranet, modem to Internet, local area network, 
10 metropolitan area network, wide area network, intranet, and wireless network. In all cases the 
client would use a protocol that is understood by the interview server over the specific 
communications network. 

[0038] Interviewing Through a Phone Device: FIG 9, Demonstrates how the interview 

15 system can be wrapped with a telephony bridge (904, 906), to support telephone based clients. 
These clients can use a regular line telephone, wireless telephone, voice telephone application on 
a computing device, or video phone using ITU H.XXX protocols. The interviews may be for 
training or real job seeking purposes. Since the interview is primarily using the media stream 
(audio and optional video), there is little dependency on the specific type of voice 

20 communications network used, other than quality of the signal and possible loss of connection. 
The computer based voice job interview will work over local telephone carriers, long distance 
carriers, wireless telephone carriers, data over Internet carriers, and other capable carriers. The 
specific network protocol of wireless carriers such as CDMA or GSM is not critical to the 
system, since the end points will use voice. The client side will initiate or receive a call from the 

25 interview server. The interview server will use telephony components to send or receive phone 
calls. Once the connection has been established, the server's telephony equipment can detect 
DTMF buttons as well as receive and transmit an audio and optional video stream. The video 
stream can come from computer generated imagery, where the server generates single images or 
multiple frames per second imagery then transmits it through the video phone call center using 

30 an audio/video IO system adapter (907). In both cases audio is generated on the server and 
streamed as an audio stream (905, 907). Input audio is received and turned into chunks of 
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discussion input and placed into the system queues for analysis by the expert systems. The user 
of the telephone client will experience a phone job interview. The user of the videophone client 
will have an experience similar that of a multimedia PC user, which is simulating a realistic job 
interview experience. 

5 

[0039] Control of a Virtual Interviewee Character: The interview system has several 

ways of having the user participate in the interview beyond that of the actual discussion. The 
interviewee can choose to use a camera to represent him or herself in the interview process. This 
still image or periodic rate video stream can be used to detect movement of the interviewee. An 

10 object identification and motion tracking system can be used to identify the background, head, 
body, and hands. To improve the capabilities of the system, the user may be asked to sit in front 
of the video camera at an appropriate distance, similar to that of an interview table, while 
simultaneously setting a helpful view and identification upper body area for the object and 
motion tracking system The video stream can also be used in a rebroadcast scenario such as 

15 when re-broadcasting a previous or real-time interview to an external party as seen in FIG 10. It 
may be desired to have a real interactive interview simulation where the interviewee is a 
character in a graphical environment with interviewers. In this case, the user can control his 
character directly or indirectly. A user may control his character by specifying a body position 
or action such as sit up, nod head, look at interviewer #2. A user may also control his 

20 expressions directly by specifying a specific emotional state such as express happiness or express 
disappointment. Indirectly, a user may configure his or her character to behave in a certain way, 
and having that automatic behavior be executed by the animation system. An example of an 
automatic behavior is asking the character to use hands when speaking at a certain intensity. 
Once specified the character will automatically use hand gestures when speaking at a frequency 

25 or intensity level previously specified by the user. A user may also specify automatic emotions, 
such as configuring a happiness level throughout the interview. During idle times, interviewees 
that are happy will automatically smile versus interviewees that are not happy will express 
disappointment. Advanced features could directly or indirectly control the animation and 
behavior that a user character portrays. For example, in a multiple interviewer interview, the 

30 user may want to directly control and focus on one interviewer, or automatically make eye 
contact with the various interviewers. The system could use a variety of methods to set the 
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virtual character controls. Internally, a collection of variables for possible actions can have 
default automatic values or specific action specific values. 

[0040] Supporting Multiple Simultaneous Interviews: In a networked environment, the 

5 interview system can support multiple simultaneous interviews. The communications system in 
FIG 6, shows how several types of clients can connect to the server at the same time. To 
simultaneously server a multitude of clients, the server can use a scheduling algorithm, polling, 
or threads. These servicing algorithms can wrap the actual process of moving data and messages 
to and from the server. For example in a near real time TCP/IP environment, the server can be 

10 notified instantly when data has arrived. In fact, the server communications subsystem (609) 
may actually be sleeping or serving other clients until there is data to be read. This is also the 
case with most modern telephony (904, 906) hardware and programming interfaces. In both 
cases, the server can perform system logic, and handle a multitude of clients simultaneously. 
There are some clients and protocols that are connectionless or do not support events, that may 

15 require periodic messages or polling. This decision may have been preferred to support some of 
a specific client's design goals, such as having the ability to work from behind a personal 
firewall. In such a case, the client application should connect to an external TCP connection or 
under more secure conditions only connect to an HTTP server. The interview server can act as 
such an end point for an interview client, and periodically service the interview client based on 

20 periodic messages. In this case the interview client will send messages using URL parameters or 
POST data. The HTTP interview client will receive messages embedded inside the request 
HTML, perhaps in XML format. Each client that is connected to the server should be uniquely 
identified by the interview server. The communications subsystem, call center, or video phone 
call center will be responsible for providing a unique client id for the connected client. At any 

25 point the interview server will have client specific session information based on the client id. 
Regardless of whether a client is actively connected at the moment, a server will be able to 
process real time interview activities and schedule outgoing messages to be sent at real time or at 
the next polling message. In a more advanced configuration multiple interview servers can serve 
a greater load in several ways. First a DNS, or service finder server can be used by clients to 

30 find an available interview server. Second, load balancing hardware can be used in front of the 
interview server which will seamlessly distribute the interview clients to an array of interview 
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servers. In both cases the interview servers can manage a client for the duration of the session 
while keeping client specific information in memory, harddrive, network storage, or database. 
The servers can also store the client specific information in a shareable location such as a 
network storage and database, which would allow multiple servers to service clients independent 
5 of a specific client/server binding. 



[0041] Multiple Forms of Simultaneous Input and Output: In the design of the interview 

system it is important to note that the system not only supports a wide range of input and output 
options. The system also supports using multiple forms of input and output at the same time. 

10 For example, a user should be able to view the closed captioning text of an interviewer as well as 
hear the voice of the interviewer character speaking. In the case of multiple interviewers, the 
closed captioning text may provide speaker information, and the speech of the interviewers may 
have different pitches. The user should be able to type to communicate, speak into a microphone 
to communicate, or speak and type to communicate. Depending on the nature of the client 

15 machine the input may be transformed locally or remotely at the interview server. An example 
of this transformation is utilizing a speech recognition system, which will produce words from a 
speaker's text. Another example of a transformation is a sound based input system, which lets 
users speak and the specific phrases are not used as input, but it allows the user to practice for an 
interview by using spoken language as a continue command. The raw audio is examined for 

20 duration, amplitude, and frequencies to detect if the audio input has qualified for real spoken 
words. In addition, the sound-based input system can be used when requiring an interview with 
raw audio, without text or speech recognition. Finally, a local machine will be able to support 
multiple inputs and multiple outputs simultaneously, when supported by the proper hardware and 
operating system. A networked machine that uses serial or parallel message streams, may queue 

25 data serially, but the local machine will be able to utilize the input and output simultaneously, 
when supported by the proper hardware and operating system. 

(00421 The Interview Discussion Engine: FIG 3, shows how the software is able to 

simulate an interview conversation based on a dynamic interview plan and a set of internal 
30 expert systems. This allows a user to experience a series of interconnected discussions that 
create an interview discussion as a whole. The system uses natural language tools to evaluate 
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speech or text input (300). A variety of processing techniques (302) can be used to identify if 
syntax, vocabulary, and grammar are valid. Although these techniques may not be able to 
validate all forms of a particular language, the system is often able to identify invalid input and 
react accordingly. The system capability to react to input is higher than a general purpose 
5 language parser because of the focus on interview discussions and supported data. Data files, 
which are created by an AI (Artificial Intelligence) Editor, provide data to the language 
processors and expert systems. Since the majority of language data (302) is separated from the 
code, the system will be able to support interviews in multiple languages such as English, 
Spanish, French, Italian, and non-Latin languages such as Chinese, Hebrew, Russian, Korean, 

10 and others. As already discussed the simulation discussion is controlled by an interconnected set 
of state machines. A specific set of state machines is initially generated based on an interview 
plan of the selected job FIG 4. These state machines (303) know how to handle specific pieces 
of a conversation such as a greeting stage, resume discussion stage, particular skill review stage, 
company discussion stage, and other stages. The states know how to transfer control to one 

15 another based on a variety of factors including the events of the current interview. Each state 
machine contains specific logic that defines how to process inputs and outputs in relation to other 
events which may have occurred during the interview. States are able to share information such 
as (304) discussion memory, (305) input data, (306) output data, and (307) session data. 
Memory may include many kinds of knowledge and information from previous interviews. 

20 Input data may include pure and processed user input, as well as other information that was 
gathered or realized about the user. Output data includes data that was spoken to the user and 
other information that was created during the interview. Session data includes communications 
information and other environment information. 

25 [0043] Configurable Language Selection: FIG 7, depicts how the system is organized in a 

specific manner to allow a wide range of languages to be supported. A language database (703) 
is used to store all general information regarding a localization. This will help identify 
synonyms, pronunciation rules, common phrases, common questions, and other general purpose 
textual resources. The job knowledge database (704) can also be altered. The job knowledge 

30 database has job specific information such as lists and values, but it also contains language 
specific textual phrases. An example of language specific job knowledge text is a job skill 
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question. The system is flexible and supports multiple languages by changing the language 

database and the job knowledge database. It is also possible to change only the language DB, 

have the job knowledge database in one base language and have a base language to target 

language component. Internally the system supports Unicode based characters, which support a 

5 multitude of languages and characters. Consideration should be made for the user interface such 

as the specific application help system. The interview system (702) may also utilize a set of 

speech recognition or speech generation components that may require either manual 

configuration or dynamic selection based on the language mode of the session. The flexible 

language support also applies to FIG 9, the call center configurations. 

10 

[0044] Administration and Integration of Live Interviewers: FIG 10, shows how human 

administrators have the ability to directly and remotely control and manage interview servers 
including the ability to act as a live interviewer thus receiving and controlling any outgoing 
speech, text, video, and characters that the user is experiencing in the interview. The interview 

15 system allows an external program (1003) to hook into the interview system logic (1006) and 
control some or all of the interview. This can be a useful application if a third party such as a 
career advisor or employer wish to interview a client remotely. The novelty of this new 
invention is that the computer generated interview system can manage all or some of the 
interview, and the administrator may passively monitor, or take control of the interview 

20 conversation using the default computer generated imagery for video if required, or completely 
replace the computer generated interviewer with his or her text, speech, video. It is possible for 
the administration program to be a local program connected to the interview server, or a remote 
program accessing the system through a network. This administration program has the ability to 
monitor and interact with several interview clients simultaneously just as the server logic handles 

25 several clients simultaneously. The administration tool will also have the ability to access 
information about the interviewer such as resume and other application data. 

[0045] Interview Result Analysis: Once the user has fully completed the interview, the 

system processes individual and collective responses qualitatively and quantitatively to provide 
30 users with analysis, compare candidates, compute rankings, estimate outcomes, provide reports, 
provide hiring recommendations. The system will use an evaluation and statistics module in the 
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discussion engine to identify trends and problems such as excessive delays while waiting for an 

answer or problems comprehending a specified percentage of input. The system will also use the 

job knowledge (406, 407) to identify scores based on answers identified within the discussion 

engine. Interview jobs reference a job description which specifies what skills are required for 

5 each job, as well as what levels of competency are required for each skill. The system will use 

this information in scoring applicants. Job descriptions also have qualitative factors such as 

traits, and although some trait questions have clear answers others do not. Sometimes the system 

will query the user as to the capabilities of a specific skill or trait and log the results of his or her 

answer. Depending on the job description, a skill may require a certain level of assurance that a 

10 user is of a certain skill level. The system will use that information to further ask questions 
about certain topics. The system will not only provide analysis, but also it will provide reports 
and any available resume, job application, transcript, audio, and video. Training applications can 
use the interview analysis to improve interview performance and the analysis can be presented in 
the form of feedback. Hiring applications or systems may use the interview analysis to match or 

1 5 screen job applicants. 



[0046] General and Job Specific Discussion Topics: FIG 4, describes how a user is able 

to choose a job (401) and how that job has concrete information that is used during an interview 
by the simulation system (408). A user can choose a job in several ways. One possible method 

20 is to have the user select a job from a set of classified ads (402). Internally, each classified ad 
will contain unique information that will correspond to a company (403), interviewer (404), 
interview plan (405), position information (406), and job knowledge (407). Companies contain 
information such as a description, number of employees, industries, culture, benefits, products, 
interview room environments, and much more. Interviewers are characters that have visual and 

25 non-visual characteristics similar to that of company managers, HR, and line managers. 
Interview plans allow for many different types of interviews, by building an interview agenda 
that drives the interview and may or may not allow for temporary or permanent deviation of the 
plan. Interview plans allow the system to have a flexible and realistic interviewing policy. 
Position information contains data relating to the description of a job, responsibilities of a job, 

30 required general skills, required job specific skills and desired qualities including those that are 
essential, optional, and extra. Position information refers to skill files and job knowledge that 
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contain discussion information that is embedded into the conversation by the expert system. 
Position information also provides a weighting for each of the position requirements, so that 
accurate final interview scores can be computed. The system also has a set of secondary skills 
and traits that may come up during interviews. These are general and behavioral questions. 
Common general topics include teamwork, goals, flexibility, creativity, initiative, and self- 
assessment. Each of these topics and many more can be available to the system, and utilized in 
any interview plan that would like to discuss general topics. In conclusion, the system has the 
capability to ask and discuss specific or general interview questions. 

[0047] Configurable Interview Scenarios: The interview system, methods of 

communications and control, and methods of interview discussion described herein have many 
uses, such as for automated interviews of applicants and interview training. FIG 7, demonstrates 
that the invention presented can also be used for a wide variety of other interviewing 
applications. Extended application interview systems of value can be created by providing new 
forms of interview type knowledge (705) in combination with implementing or adjusting any 
necessary user interface elements (701). For example, the system can be extended to support 
school admissions interviews, visa application interviews, and performance arts auditions and 
interviews. So in conclusion, while specific embodiments of the invention have been disclosed 
in detail, it will be appreciated by those skilled in the art that many modifications and 
alternatives may be made without deviating from the spirit and scope of the invention defined in 
claims. 
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